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Abstract 

Two hundred and nine children receiving early childhood special education services for 
developmental disabilities or delays who also had behavioral, social, or attentional difficulties 
were included in a study of an intervention to increase school readiness, including early literacy 
skills. Results showed that the intervention had a significant positive effect on children’s literacy 
skills from baseline to the end of summer before the start of kindergarten (d = .14). The 
intervention also had significant indirect effects on teacher ratings of children’s literacy skills 
during the fall of their kindergarten year (f = .09). Additionally, when scores were compared to 
standard benchmarks, a greater percentage of the children who received the intervention moved 
from being at risk for reading difficulties to having low risk. Overall, this study demonstrates 
that a school readiness intervention delivered prior to the start of kindergarten may help increase 
children’s early literacy skills. 
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Increasing Pre-Kindergarten Early Literacy Skills in Children with Developmental Disabilities 
and Delays 
Efforts to ensure that young children with developmental disabilities and delays receive 

early intervention have been central to federal education legislation for the past several decades 
and have stimulated the development of a range of Early Childhood Special Education (ECSE) 
services and supports (Gargiulo & Kilgo, 2011). One of the major goals of such early 
intervention is to reduce the need for special education services once children enter school, thus 
readying them to enter general education (Phillips & Meloy, 2012). For all children, school 
readiness skills are critical in determining academic and social success through the subsequent 
school years and even into adulthood (Campbell et al., 2008; Schweinhart et al., 2005). Among 
school readiness competencies, early literacy skills that form the foundation for reading may be 
particularly important to academic achievement (Torgesen, 1998) which is linked to social and 
behavioral adjustment in school (Bennett, Brown, Boyle, Racine, & Offord, 2003; Halonen, 
Aunola, Ahonen, & Nurmi, 2006). The promotion of those skills may thus be important to school 
psychologists working to prevent future difficulties for students. This study describes the results 
of an evaluation of the Kids in Transition to School (KITS) Program, an intervention designed to 
prepare children with developmental disabilities and delays with concurrent behavioral, social, or 
attentional problems for the transition to kindergarten. The KITS Program promotes critical early 
literacy skills among other school readiness abilities. (Because behavioral, social, and attentional 
problems all represent some difficulty with behaviors, we collectively refer to them as 
“behavioral problems”.) 


Early Literacy and School Readiness 


Early literacy skills, such as knowing one’s letters, letter sounds, and basic concepts 
about print, are the foundational skills for reading and represent an important component of 
school readiness. Children who begin school without these skills may struggle academically and, 
once they fall behind their peers, find it increasingly difficult to catch up (Torgesen, 1998). Early 
literacy skills positively predict (rs = .44 - .47) both reading and math achievement through 
elementary and middle school (Duncan et al., 2007). Poor reading ability has further been linked 
to behavioral difficulties at school (Halonen et al., 2006), and the long-term consequences of 
poor literacy include school drop-out, involvement in antisocial activities, and poor educational 
and occupational attainment (Bennett et al., 2003; Maughan, Gray, & Rutter, 1985). In the 
current study, we focused on several early literacy skills, including phonological and phonemic 
awareness, letter naming, and understanding of concepts about print. Phonological awareness 
(e.g., the understanding that words are comprised of a variety of sounds) and phonemic 
awareness (e.g., the ability to identify the distinct sounds in a word) are two of the strongest 
predictors of early reading skills (Schatschneider, Fletcher, Francis, Carlson, & Foorman, 2004). 
The ability to correctly identify letters is also a robust predictor of reading outcomes (National 
Institute for Literacy, 2009; Schatschneider et al., 2004). In addition, understanding concepts 
about print (e.g., left to right orientation, differentiation of words from pictures) is key to reading 
comprehension (National Research Council, 1998) and is associated with other early literacy 
skills such as phonological awareness (Levy, Gong, Hessels, Evans, & Jared, 2006) and letter 
identification (Diamond, Gerde, & Powell, 2008). 

Early Literacy Skills, Developmental Disabilities and Delays, and Behavior Problems 

Despite the critical importance of early literacy skills to reading and other academic 


outcomes, there is very little literature on these skills in children with developmental disabilities 


and delays prior to school entry. One of the few studies of kindergarten readiness in children 
with developmental disabilities and delays found that 58 — 96% of the children, depending on 
disability category, were not demonstrating the skills that a child transitioning to kindergarten 
would need to be successful, and thus were “not ready” for the transition (Lloyd, Irwin, & 
Hertzman, 2009). Further, 62% of those children were performing below academic standards in 
reading when they were assessed again at fourth grade (Lloyd, et al., 2009). A better 
understanding of the early literacy skills of this group of vulnerable children is critical in order to 
help them to achieve later school success. 

Among children with developmental disabilities and delays, those with behavioral or 
social difficulties are at the highest risk for a range of poor outcomes including low academic 
performance, low educational attainment, and early involvement in criminal activities (Trout, 
Nordness, Pierce, & Epstein, 2003). A number of studies, both in typically developing children 
and those with developmental disabilities, have found associations between reading ability and 
problematic behavior (Anderson, Kutash, & Duchnowski, 2001; Gray, Carter, Briggs-Gowan, 
Jones, & Wagmiller, 2014; King, Lembke, & Reinke, 2015; Morgan, Farkas, Tufis, & Sperling, 
2008). Children with early behavioral difficulties, particularly inattention, are more likely to 
experience difficulties with early reading (Gray et al., 2014). Conversely, early reading 
difficulties appear to elevate a child’s chances of developing problem behaviors over time 
(Morgan et al., 2008). 

Particularly worrisome for children with developmental delays and behavioral difficulties 
are findings that deficits in skills essential to reading appear to increase with the severity of 
behavioral and/or social problems (Trout, Epstein, Nelson, Synhorst, & Hurley, 2006). Over 


time, while reading skills improve in children with disabilities but without behavioral problems, 


children with concurrent behavioral issues do not show such gains (Anderson et al., 2001). Thus, 
children with developmental disabilities and behavioral problems may start school with worse 
skills than their peers and their resulting difficulties in reading may then exacerbate their 
behavioral problems. Increasing foundational reading skills through early intervention could help 
to prevent escalating academic and behavioral problems in these children. 
Improving Early Literacy Skills 

Although children with developmental disabilities and delays by law receive services to 
remediate those delays, only relatively recently has there been a move to make early literacy 
skills a specific focus of these services (Carta & Kong, 2007). Head Start programs are mandated 
to include children with developmental disabilities and delays and the recent Head Start Impact 
Study found that these children appear to benefit from Head Start in math and socio-emotional 
development (U.S. Department of Health and Human Services, Administration for Children and 
Families, 2010). Children with developmental disabilities and delays who attend universal and 
inclusive pre-kindergarten programs show increases in early literacy similar to those of their 
peers without disabilities (Phillips & Meloy, 2012). However, not all children with 
developmental disabilities and delays will have opportunities to attend Head Start or pre- 
kindergarten programs, as spaces in Head Start and other preschool programs are becoming 
increasingly limited and universal pre-kindergarten is not yet widely available (Annie E. Casey 
Foundation, 2014). 

Perhaps one of the largest missed opportunities for many early intervention and early 
education programs is that many of them run on an academic year calendar. This creates a 
summer services gap during which the most vulnerable, highest-risk children may lose or fail to 


gain valuable skills (Alexander, Entwisle, & Olson, 2001), and children with special education 


needs may experience the largest losses (Allinder & Eicher, 1994; Patton & Reschly, 2013). 
Additionally, the transition into school seems to be an optimal time to intervene on school 
readiness (Pianta & Cox, 1999). As children’s school readiness skills can make a critical 
difference to their academic and social progress (Duncan et al., 2007), programming to augment 
those skills and to fill any services gap just prior to school entry seems warranted, particularly 
because children with developmental disabilities and delays and behavioral difficulties are at 
such high risk for academic failure. 

The KITS Program is a short-term, intensive school readiness intervention designed to 
fill the summer services gap and augment the school readiness skills of children at high risk for 
difficulties with academic and social adjustment to school. A central premise of the KITS 
Program is that through improving school readiness skills at the beginning of kindergarten, 
subsequent school functioning will be improved. To this end, one of the primary foci of the 
program is on developing early literacy skills such as letter recognition, phonological awareness, 
and concepts about print. Strengthening these skills just prior to school entry is likely to have 
positive effects since, as is noted above, early literacy skills are linked to better reading 
proficiency across the later school years (Duncan et al., 2007). The KITS Program also 
emphasizes increasing children’s self-regulatory skills. These skills, which include children’s 
ability to inhibit inappropriate responses (such as shouting out answers instead of raising their 
hands), to direct and sustain their attention, and to regulate the intensity of their emotional 
responses, have been linked to better reading later in school and even success in college (Blair & 
Razza, 2007; Hooper et al., 2011; McClelland, Acock, Piccinin, Rhea, & Stallings, 2013). Thus, 
the KITS Program works to promote positive school adjustment and longer-term reading 


achievement by increasing readiness skills. 


As is noted above, the timing of the KITS Program is a key feature of the intervention. It 
is delivered during the summer before the transition to kindergarten (the school readiness phase) 
and the first 2 months of kindergarten (the transition/maintenance phase). Thus, KITS fills a 
critical summer services gap for many children and augments early intervention and ECSE 
services that they may have previously received. Another important feature of the KITS Program 
is its ecological validity. The program is conducted in a classroom with typical kindergarten 
rules and routines. This creates an opportunity for children to practice the skills and behaviors 
needed in kindergarten within an environment with a low student to teacher ratio (specifically 
4:1 for the program). Thus, teachers can scaffold and positively reinforce children’s use of their 
new skills. A third important feature of the intervention is the provision of high-density learning 
opportunities, meaning that as much time as possible within the curriculum is focused on 
learning activities specifically designed to enhance critical school readiness skills. Within many 
typical early learning settings, children might spend less than half of their time in instructional 
activities (Hamre & Pianta, 2007). 

The KITS Program was originally tested in a randomized clinical trial with children in 
foster care, a population with high rates of developmental disabilities and delays as well as 
behavioral difficulties (Lightfoot, Hill, & LaLiberte, 2011; Slayter & Springer, 2011). Results 
from that trial showed significant positive effects on the children’s literacy (including letter 
naming, phonemic awareness, and understanding of concepts about print) immediately before 
entry into kindergarten (Pears et al., 2013). Given the success of the program with a population 
that overlaps that of children with developmental disabilities and the need for programs to 
prepare children with developmental disabilities for school, a full-scale randomized efficacy trial 


of the program with these children was conducted. Results of the evaluation of effects on 


children’s early literacy skills are presented within the current study. The KITS Program also 
includes foci on self-regulatory and prosocial skills. In order to be able to examine multiple 
timepoints and informants, we have chosen to separately present intervention effects on different 
foci. A study of the intervention effects on self-regulation has shown positive impacts into 
kindergarten (Pears, Kim, Healey, Yoerger, & Fisher, 2015) for the same sample used in this 
study. 

The current study had three goals. The first goal was to examine whether the KITS 
Program had direct effects on children’s early literacy skills just prior to kindergarten entry, 
controlling for baseline skills. Based on results with children in foster care, we hypothesized that 
the intervention would have a significant positive effect on early literacy skills. The second goal 
was to examine the premise that by improving school readiness prior to kindergarten entry, 
kindergarten academic outcomes would be improved. Thus, we hypothesized that the 
intervention would indirectly positively affect literacy skills in kindergarten through earlier 
direct effects on pre-kindergarten skills. 

There are a number of recent findings to suggest that we would find more modest effect 
sizes than those demonstrated in studies of longer-term interventions and interventions with 
typically developing children. Specifically, in a meta-analysis of the effects of summer 
interventions for general and high-risk students, Cooper and colleagues (2000) found a median 
effect size of .19, which would be considered modest according to standard measures of effect 
sizes (Cohen, 1988). Further, the study by Cooper and colleagues largely included typically 
developing children. An efficacy trial of a summer literacy program that specifically examined 
effects sizes for children with developmental disabilities found that those effect sizes were as 


much as 20% lower than those for typically developing children (Zvoch & Stevens, 2015). Thus, 
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there was a potential for the effect sizes found in the present study to be more modest than those 
seen with other children. 

This led to our third study goal which was to answer recent calls to complement 
statistical representations of intervention effect sizes with descriptive information on intervention 
effects, including for example, changes in risk relative to standard benchmarks (Lipsey et al., 
2012). Thus, we examined how the KITS Program affected the children’s risk for reading failure 
as measured by benchmarks based on the general population. We hypothesized that the 
intervention would be associated with greater increases in the percentages of children considered 
to be at low risk for reading difficulties. However, since the benchmarks were created for the 
general population of children, we did not expect the intervention to bring all of the children into 
a low-risk group. 

Method 
Participants 

Two hundred and nine children and their families were recruited in 4 yearly cohorts of 
50-55 children and their families through the county public agency responsible for Early 
Intervention and Early Childhood Special Ed (EI/ECSE) services for young children. Each year, 
the parents of all of the children transitioning to kindergarten received a letter explaining that 
their children would be screened for study participation and were given opportunities to opt-out 
of the process. To be eligible for study participation the child had to be receiving ECSE services 
and to have been rated to have behavioral difficulties as follows: First the children’s ECSE 
service coordinators completed an initial rating questionnaire on which they indicated whether 
the child had behavioral, social, and/or attentional difficulties that would interfere with his or her 


adjustment to kindergarten (scored “1” if the child had such difficulties). Second, if the child had 
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such difficulties, the service coordinators completed three subscales of the Early Screening 
Project (ESP; Walker, Severson, & Feil, 1995), an empirically-based screening measure for 
significant behavior problems in 3- to 6-year-olds. The scales included the 16-item Critical 
Events scale (measuring the occurrence/nonoccurrence of events indicative of behavior 
problems; e.g., “steals,” “is self-abusive’’), the 9-item Aggressive Behavior scale (measuring 
frequency of aggressive behaviors; e.g., “physically assaults adults,” “has tantrums’’), and the 9- 
item Maladaptive Behaviors scale (measuring frequency of antisocial and nonsocial behaviors; 
e.g., “refuses to participate in games and activities with other children during free (unstructured) 
play,” “behaves inappropriately in class when directed (shouts back, defies teacher, etc.)). Prior 
research employing the ESP has found high interrater and test-retest reliability and concurrent 
validity with other measures of aggressive, disruptive, and externalizing behaviors (Feil et al., 
2005; Feil & Walker, 1995; Feil, Walker, Severson, & Ball, 2000). Clinical cut-off scores 
indicate if the child is considered to be at risk for behavior problems and a measure of the overall 
risk severity may be calculated by totaling the number of subscales (O—3) on which the child 
exceeds the at-risk clinical cut-off score. Overall risk severity correlates positively with other 
measures of externalizing behaviors (Feil et al., 2000). For each child, an overall risk severity 
score was calculated from their ESP scores. This overall risk severity score was then summed 
with the service coordinators’ initial rating of the child’s difficulties to produce a final behavioral 
risk score. All children with a final behavioral risk score of 1 or greater were eligible for the 
study. 

Exclusion criteria for the study included the following: hearing or vision impairment that 
would limit participation in the assessment or intervention protocols, IQ below 70, not being a 


monolingual or bilingual English speaker, being in a foster placement, or receiving full-time 
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(12+ hours/week) ECSE services in the summer (since most of these families opted for the ECSE 
services over the KITS Program and KITS is designed to fill a summer services gap). 

A project staff member set up a home visit with eligible families to explain the project 
and obtain informed consent. Figure 1 shows the flow of participants through the recruitment and 
randomization process. Of the 392 eligible families, 18 were found to be ineligible when they 
were contacted (i.e., they did not meet the criteria for eligibility but that was not apparent from 
the information that we had received at screening; for example, in some cases, families had not 
told their ECSE coordinators that they were not planning to enroll their children in kindergarten 
in the fall). Of the remaining families, 225 (60%) agreed to participate. After parental consent 
was obtained, the families were randomized to the KITS intervention or to the services-as-usual 
(SAU) group. There were no significant differences on child gender, age, ethnicity, disability 
diagnosis, or final behavioral risk score between the families who agreed to participate and those 
who chose not to participate or respond. Of the 225 families who initially agreed to participate, 
16 dropped from the study before they completed any of the assessments. This left 209 
participating families. Families who dropped out of the study were not significantly different 
than those who continued to participate on child gender, age, ethnicity, disability diagnosis, or 
final behavioral risk score. There were no significant differences in the attrition rates between the 
KITS and SAU groups. 

The demographic characteristics and the mean scores on the ESP for the 209 participating 
children and families (KITS group = 107; SAU group = 102) are shown in Table 1. There were 
no significant differences between the two groups on any of the variables. The larger proportion 
of boys in the sample was expected, as the rates of behavioral difficulties are generally higher in 


boys (American Psychiatric Association, 1994). 
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Study Design and Procedures 

Data collection procedures. The children and their parents visited the research center to 
complete a 60-90 minute interview at the beginning of the summer prior to the start of any 
intervention activities (Time | [T1]) and at the end of the school readiness phase of the 
intervention just prior to the start of school (Time 2 [T2]). Children completed standardized tests 
and parents completed semi-structured interviews and questionnaires. Teachers completed 
questionnaires approximately 2 months after the children began school at the end of the 
transition/maintenance phase of the intervention (Time 3 [T3]). Only teacher data could be 
utilized for outcomes at T3 because, in order to reduce assessment burden, neither children nor 
their parents completed measures at this timepoint. All data collection procedures were approved 
by the Institutional Review Board of the research center at which the study was conducted. 

Intervention protocol. The KITS intervention occurred during the 2 months prior to 
kindergarten entry (school readiness phase) and the first 2 months of kindergarten 
(transition/maintenance phase). The intervention consisted of two primary components: a 24- 
session school readiness group for the children (2 hours twice weekly in the school readiness 
phase, 16 sessions; 2 hours once weekly in the transition/maintenance phase, 8 sessions) focused 
on promoting early literacy and social-emotional skills and an 8-session parent group (2 hours 
every 2 weeks; 4 sessions in each intervention phase) focused on promoting parent involvement 
in early literacy and school. Morning and afternoon group sessions were available. Families 
picked the most convenient time slot and then attended all groups at that time. 

School readiness group structure and curriculum. Similar to a typical kindergarten 
schedule, the school readiness group sessions had a highly structured, consistent routine with 


many transitions between activities. Groups were held in center- or school-based classrooms. 
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The manualized school readiness group curriculum covered three skill areas identified in prior 
research as being associated with later school outcomes: early literacy skills (e.g., letter names, 
phonological awareness, conventions of print, and comprehension), prosocial skills (e.g., 
reciprocal social interaction, social problem-solving, and emotion recognition), and self- 
regulatory skills (e.g., handling frustration and disappointment, controlling impulses, following 
multistep directions, listening, and making appropriate transitions). The curricular objectives 
were clearly specified for each session by skill domain. Skills were introduced at circle time 
lessons. The daily activities (e.g., art projects, dramatic activities) were designed to practice the 
session skills. For example, the early literacy activities included a letter of the day, a poem of the 
week, and storybook and dramatic activities. For the letter of the day activities, children were 
introduced to a new letter of the day, the teachers helped the children to produce the sound of the 
letter, children indicated whether a pictured object began with the letter and then produced their 
own words starting with the sound, thus reinforcing letter naming and letter-sound knowledge. 
Subsequent early literacy activities then incorporated that letter. The teachers and children also 
read the poem of the week together, with teachers asking children questions to reinforce concepts 
about print such as: “Where should I start to read?”. To increase phonological awareness, 
teachers encouraged children to find the words that rhyme. Children also searched within the 
poem for instances of the letter of the day. A lead teacher and two assistant teachers conducted 
the school readiness groups that consisted of 12—15 children. The low child-to-staff ratio allowed 
the teachers to provide children with high levels of support and feedback. 

Parent group structure and curriculum. The parent group meetings were held at the 
same time as the school readiness groups, and were led by a facilitator and an assistant. The 


manualized parent curriculum included foci on skills relevant to the kindergarten transition (e.g., 
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helping children to develop their early literacy skills, developing routines around school 
activities, preparing children for the kindergarten transition, home-school collaboration, and 
behavior management skills). Each group session focused on a specific topic although sessions 
built on topics from previous weeks. The facilitators presented information, led structured group 
discussions of the materials, and addressed questions and concerns. Skill acquisition was 
reinforced via role-plays and opportunities to practice. Because parents were often working 
during the day and would thus not be able to participate in the groups, we provided curriculum 
content during home visits (or a phone call if necessary) for any groups that parents missed. In 
order to remove potential barriers to attendance (such as the need for child care for siblings of 
the participating child), we offered free child care, food, and aid with transportation to parents on 
days that the parent group met as well as a raffle at each group meeting for gift cards to local 
stores. 

The children and parents received supplemental materials to support the implementation 
of new skills. These included weekly homework assignments for the children, home practice 
activities for the parents, and Home—School Connection newsletters outlining the school 
readiness group topics for the week. At weekly intervention team meetings, the progress of 
individual families within the three school readiness domains was discussed, and strategies to 
address any specific literacy or behavioral needs within the broader curriculum were planned. 

Intervention staff training. The KITS school readiness group teachers and parent group 
facilitators completed a standardized 40-hr training program. The curriculum and the behavior 
management strategies used by the school readiness group teachers were manualized as was the 
parent group curriculum. Teachers and facilitators were consistent across sessions of the school 


readiness and parent groups within a cohort except in cases of unavoidable absences. The 
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standardization of the KITS Program curricula, training, and teaching and behavior management 
models allowed for training multiple teachers who could implement the curriculum with high 
fidelity (see fidelity of implementation section below). 

Fidelity of implementation. Implementation fidelity for both the school readiness 
groups and the parent workshops was determined in vivo or via videotape by coding the presence 
or absence of key elements of the curricula. All of the school readiness groups and parent 
workshops were coded, except in cases in which the groups were not videotaped due to either 
mechanical or human error. Coders were trained on the curriculum and elements of behavior 
management and group process by the lead trainer for the program. For the school readiness 
groups (n = 188), the teachers covered all the curricular elements (e.g., introducing the letter of 
the day, presenting the correct social skill) 98% (range = 50—100%) of the time. In the parent 
workshops (n = 55), the curricular elements (e.g., presenting the correct topic of the day, asking 
for examples of child behaviors when indicated) were covered 92% (range = 67—100%) of the 
time. Additionally, coders rated whether the teachers and parent workshop facilitators 
implemented key behavior or group management strategies (e.g., “Pre-taught expectations”, 
“Ignored or re-directed child non-compliance” for the playgroups and “engaged parents in 
conversation” and “redirected conversations when necessary” for the parent workshops) on a 3- 
point scale: 1 ‘did not occur’, 2 ‘sometimes occurred’, and 3 ‘regularly occurred’. On average, 
teachers received a rating of 2.94 (range = 2.13—3.00) and the parent group facilitators received a 
rating of 2.85 (range = 2.33—3.00), indicating that they used the behavioral management 
strategies almost all of the time. 

Twenty percent of the sessions were double-coded in order to compute inter-rater 


reliability for the implementation fidelity coding. Other studies using observations to code 


Vy 


implementation fidelity have double-coded approximately 10-16% of cases (Cross et al., 2015; 
Hansen, Pankratz, & Bishop, 2014). For the content specific implementation fidelity (whether 
specific curricular elements were covered), the average percent agreement across all items was 
98% for the school readiness groups and 96% for parent groups. For the implementation fidelity 
checklists indicating whether strategies and processes were used correctly, the average weighted 
percent agreement (because the scale had 3 points) was 96% for the school readiness groups and 
91% for the parent groups. 

Attendance. School readiness group attendance, parent workshop attendance, and home 
visits (or phone calls if a parent refused a visit) to deliver missed workshop curriculum were 
tracked. On average, the children attended 62% of the school readiness groups (range = 0— 
100%), and the majority (60%) attended 60% or more of the groups. These rates are comparable 
to those for other summer programs for high-risk children (August, Lee, Bloomquist, Realmuto, 
& Hektner, 2003). For parent workshops, the primary variable of interest was the amount of 
curriculum received since we understood that not all parents would be able to attend groups due 
to work schedules and other commitments and would receive the curriculum via home visits. On 
average, the parents received 54% of the curriculum (range = 0-100%), and the majority (52%) 
received 60% or more. These rates are comparable to those in other studies of short-term 
parenting interventions for high-risk samples (e.g., Begle, Lopez, Cappa, Dumas, & de Arellano, 
2012). 

Services-as-usual group. Children in this group received services typically offered to 
children with developmental disabilities, including ECSE services and evaluation, individual and 


family therapy, and participation in early childhood education and care programs. No attempt 
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was made to influence the type or amount of services. Groups did not differ on the numbers of 
children receiving services (other than KITS) during the summer. 
Measures 

For the following measures, all Cronbach alphas for scales were calculated using the 
study sample. Estimates for test-retest, split-half, and alternate form reliability were drawn from 
other studies. For all reliability statistics, Cronbach alpha values over .60 are considered to be 
acceptable (George & Mallery, 2003). 

Pre-kindergarten literacy skills (T1 and T2). Letter naming and phonemic awareness 
were measured using the Letter Naming Fluency (LNF) and Initial Sound Fluency (ISF) subtests 
of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002), 
respectively. For the LNF subtest, the total score is the number of letters correctly identified 
from a randomly ordered array of uppercase and lowercase letters in 1 minute. For the ISF 
subtest, the score is the number of correctly produced initial sounds of a word corresponding to a 
picture. Alternate form reliability for both subtests is good in other studies (r = .88 for the LNF 
and .72 for the ISF; Good et al., 2003). Further, both subtests are predictive of reading skills at 
the end of kindergarten, first, and second grades (Burke, Hagan-Burke, Kwok, & Parker, 2009). 

Understanding of concepts about print was measured using the 24-item Concepts About 
Print Test (CAPT; Clay, 2000), which assesses such print conventions as reading left to right, 
matching spoken to written words, and distinguishing pictures from text. The children received 1 
point for each correct answer, summed to produce a total score (as = .82 and .80 at Tl and T2, 
respectively). The children’s raw scores were also converted into stanine scores using the norms 
for children aged 5 to 5% years (Clay, 2005). To minimize missing data, children who were not 5 


at the time of testing were given stanine scores using the norms for 5 to 5% year olds as they 
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were all within 4 months of their 5th birthdays. This test demonstrated acceptable test-retest 
reliability in other studies (average = .91; Holliman, Hurry, & Douetil, 2010) as well as strong 
associations with other concurrent and longitudinal measures of reading skills (Holliman et al., 
2010; Stuart, 1995). 

The final indicator of pre-kindergarten literacy skills was a parent rating of prereading 
skills. The parents were asked if the child could: recognize the letters of the alphabet (0 [none] to 
3 [all]) and write his or her first name (0 [no] or 1 [yes]). Parent responses across these two 
questions were significantly, positively correlated at Tl (r= .41, p < .001) and at T2 (r= .22, p< 
.01). Although the correlations were in the moderate to low range (Cohen, 1988), the items were 
felt to have face validity. Thus, responses were standardized and averaged to produce a 
composite parent rating of prereading skills at Tl and T2 with higher scores indicating greater 
reading skills. These items have previously been used in a national survey of kindergarten 
readiness (National Center for Education Statistics, 1993). 

Kindergarten literacy skills (T3). The children’s literacy skills in the fall of 
kindergarten were measured via the teacher report on the 26-item Pre-Literacy Rating Scale 
(PLRS) from the Clinical Evaluation of Language Fundamentals Preschool—Second Edition 
(CELF-P; Wiig, Secord, & Semel, 2004). The PLRS measures the frequency with which children 
display a number of critical emergent reading and writing skills and is designed to be used with 
children aged 3 through 6 years. The teachers were asked to rate the frequency with which each 
child displayed the behaviors on a 4-point scale: 1 (never) to 4 (always) or N/A. The items were 
used to create mean scores for each of the two subscales of the PLRS: Emergent Reading Skills 
(a = .92) and Emergent Writing Skills (a = .94). In other studies, the combined subscales of the 


PLRS showed acceptable split-half reliability across ages (mean r =.97; Wiig et al., 2004), as 
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well as concurrent associations with direct assessments of children’s literacy skills (Cabell, 
Justice, Zucker, & Kilday, 2009). 

Determining level of risk for reading difficulties. The percentages of children at risk 
for reading difficulties as measured against the benchmarks for the DIBELS and CAPT were 
calculated. The kindergarten fall benchmarks for the DIBELS were utilized (Good, Wallin, 
Simmons, Kame’enui, & Kaminski, 2002) as the children in this study were about to enter 
kindergarten. Children who score 7 or below on the ISF or LNF measures are considered to have 
“some risk” or to be “at risk” for later reading difficulties while children scoring 8 or above are 
considered to be “low risk” (Good et al., 2002). For the CAPT, children in the lowest 2 stanines 
(at or below the 10 percentile) are considered to be at risk for reading failure (Clay, 2005). For 
the PLRS, it can be determined if children met a “critical score” for literacy for their age by 
summing across all of the items and comparing the resulting score to the correct benchmark for 
the child’s age. 

Intervention status. In these analyses, intervention status was represented by a 
dichotomous variable: 1 (KITS intervention group) or 0 (SAU group). 

Control variables. All of the control variables were measured at T1. As this study was 
focused on literacy skills, in order to account for any delays in nonverbal cognitive abilities that 
might contribute to school difficulties, we used the scaled score for the Block Design subscale of 
the Wechsler Preschool and Primary Scales of Intelligence-Third Edition (WPPSI-III; Wechsler, 
2002). This is one of the subscales most strongly correlated with the full-scale intelligence score 
on the WPPSI-III (Wechsler, 2002) and a widely used indicator of nonverbal cognitive skills 
(DeThorne & Schaefer, 2004). We measured but did not include the vocabulary subscale as it 


was highly intercorrelated with the measures of early literacy and might have created problems 
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of multicollinearity. Rather than forego a control for cognitive ability, we chose to account for 
nonverbal ability by using this subscale. To control for the possibility that children who received 
more EI/ECSE services might score better on measures of early literacy, two measures of 
services received were calculated from each child’s service records across the preschool years. 
All the EI/ECSE services that the child was scheduled to receive as part of his or her individual 
and family service plan were coded into five categories: individual therapy or instruction, group 
therapy or instruction (including specialized preschool classrooms), coaching and consultation to 
the family or teachers, observation, assessment or screening, and supervision and case 
coordination. Within each category, the time spent in each activity was summed to produce a 
total of service hours within each category. For this study, the totals for time spent in individual 
therapy, group therapy, and coaching/consultation were used, as these were the most direct 
services and most likely to influence school readiness. These numbers do not reflect the hours of 
services actually received but the hours that the child was supposed to receive. 
Data Analysis 

First, we examined potential group differences on the demographic and outcome 
variables and potential cohort effects. Second, the correlations among the variables were 
examined. Third, SEM using Mplus version 6.12 (Muthén & Muthén, 1998-2010) was employed 
to test the direct effects of the KITS Program on pre-kindergarten literacy, and the indirect 
effects on literacy skills in the fall of kindergarten. In all models, we used maximum likelihood 
estimation (MLR) with robust standard errors that are known to be robust to non-normality and 
non-independence in the data (Muthén & Muthén, 1998—2010). To confirm the adequacy of the 
hypothesized indicators of the latent variables for literacy, a preliminary confirmatory factor 


analysis (CFA) was performed. The CFA model included four indicators - the two DIBELS 
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subscales, the CAPT, and the parent ratings of pre-reading, for two latent variables representing 
early literacy at Tl and T2. The parameters of the early literacy indicators were constrained to be 
equal across T1 and T2 and within each timepoint, these indicators were allowed to covary. The 
model also included covariances between the Tl CAPT with the T2 DIBELS indicators, the T2 
CAPT with the Tl DIBELS indicators, and the T2 CAPT with the T2 parent rating. The fit of the 
model was satisfactory, with a chi-square of 8.82, df= 13, p = .79, CFI = 1.00, TLI = 1.01, and 
RMSEA = .02, 90% CI [0.00, 0.05]. It should be noted that TLI is considered as a non-normed fit 
index and on occasion it may be larger than 1. Values larger than .95 are considered to indicate 
good model fit (Tanaka, 1993). 

All of the four indicators loaded significantly on the early literacy latent variable. Next, 
the direct effects of the covariates on the T1 outcome, the direct effect of intervention status on 
the T2 outcome, and the indirect effects of intervention status on T3 outcomes were modeled in 
the final analyses described below. To examine potential indirect pathways from the treatment to 
the T3 outcomes, the significance of the indirect path from intervention to kindergarten literacy 
skills through pre-kindergarten skills was tested using the MODEL INDIRECT function in 
Mplus (Muthen, 2011; Preacher & Hayes, 2008). This test allows us to examine whether pre- 
kindergarten skills indeed carry the intervention effects on kindergarten literacy skills (Preacher 
& Hayes, 2008). The resulting estimate is based on a two-tailed z-test and thus the absolute value 
of 1.96 or greater would indicate that the indirect effect is significantly different from zero. The 
size of intervention effects for the T1 to T2 paths was computed using the formula for the 
independent-groups pretest-posttest design (Feingold, 2009): d = (M change-Kits/SD raw(pre-KITS)) — 
(M change-sAu/SD raw(pre-SAU)), Where M change-Kits 18 the mean change for the KITS group, M change- 


sau is the mean change for the SAU group, SD raw(pre-Krrs) 18 the pretest SD for the KITS group, 
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and SD taw(pre-sAu) is the pretest SD for the SAU group. To calculate the size of the intervention 
effect, we averaged the standardized means of all the indicators for the early literacy latent 
variable at each time point for each group. We utilized the means of the unadjusted measured 
variables because means of latent variables are not typically reported in the literature and they 
are adjusted by all of the covariates in the model. Thus, use of the measured variables was felt to 
be a more conservative approach to calculating the effect size. 

Complete data were available on all control variables for 99.0% of the sample; 0.5%, 
3.8%, and 3.8% of the sample had missing data on one or more of the indicators of the latent 
outcome variable at T1, T2, and T3, respectively. Although the missing data were minimal, Full 
Information Maximum Likelihood (FIML) estimation was utilized to take advantage of all 
available data. This method has been shown to provide unbiased estimates when data are missing 
at random (Arbuckle, 1996). The Little’s Missing Completely at Random tests were not 
significant for the model (y7 = 46.90, df = 46, p = .44), indicating a random distribution of the 
missing data. 

Results 

Descriptive Analyses 

The children’s mean scores on the measures are presented in Table 2 in the original test 
metric. There were no significant differences between the groups on the literacy measures at T1. 
For the control variables, there were no significant differences between the groups except that 
children in the KITS group had received more hours of individual ECSE services during the 
preschool years than had children in the SAU group (f = -2.08, p < .05). There were no 
significant differences based on cohort for either demographics or outcomes. The correlations 


between the pre-kindergarten literacy measures, the teacher-reported kindergarten literacy 
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measures, and the control variables (Table 3) indicated that there were significant positive 
correlations between the early literacy variables at each timepoint as well as across timepoints. 
The measures of kindergarten literacy were also significantly positively correlated with the early 
literacy measures at T1 and T2, with the exception that teacher ratings of emergent writing were 
not significantly associated with T1 initial sound fluency. As expected, the children’s scores on 
the WPPSI-III block design subscale, used here to measure non-verbal cognitive ability, were 
positively correlated with all of the literacy measures. Interestingly, neither hours of individual 
ECSE nor hours of ECSE consultation and training services were significantly associated with 
literacy with the exception of a significant negative association between individual ECSE 
services and teacher ratings of emergent writing. 
Intervention Effects on Pre-Kindergarten and Kindergarten Literacy Skills 

SEM analyses were performed to examine direct intervention effects on pre-kindergarten 
literacy skills, as well as indirect effects on kindergarten literacy skills through effects on pre- 
kindergarten skills. As described above, the two DIBELS subscales, the CAPT, and the parent 
ratings of pre-reading were used as indicators of a latent variable representing pre-kindergarten 
literacy skills. At T3, two indicators--teacher ratings of reading skills and teacher ratings of 
writing skills--formed a latent variable for kindergarten literacy skills. The children’s general 
cognitive ability, hours of individual services, and hours of consultation or training were used as 
control variables. The hours of group therapy variable was not significantly correlated with the 
outcome variables in preliminary correlational analyses, so was not included in the model. 

The model shown in Figure 2 fit the data well, with a chi-square of 71.93, df= 64, p = 
23, CFI = .99, TLI = .99, and RMSEA = .02, 90% CI [0.00, 0.05]. All of the indicators loaded 


significantly on their respective latent variables. As predicted, there was a significant 
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intervention effect on pre-kindergarten literacy skills at T2 (d = .14). Additionally, pre- 
kindergarten literacy skills had direct and positive effects on kindergarten literacy skills. Of the 
control variables, only general cognitive ability had an effect on pre-kindergarten literacy skills. 
(When we tested the same model described with the number of group service hours, that variable 
was not significantly associated with teacher ratings. Thus, to test the most parsimonious model, 
that path was not included in the final model.) To test the hypothesis that the intervention has 
indirect effects on kindergarten literacy skills, the significance of the path from the intervention 
to kindergarten literacy through T2 pre-kindergarten literacy was tested. This was a significant 
effect (B = .09, z = 3.06, p < .01). 

We compared the percentages at Tl and T2 of children in each group who were at low 
risk for reading difficulties as measured by the DIBELS LNE and ISF subscales and the CAPT 
using the standard benchmarks described above. As shown in Table 4, for the DIBELS LNF 
subscale, there was a 7% increase in the number of children at low risk for reading difficulties in 
the KITS group across the summer, while for the SAU group, there was a 4% decrease (or a 0% 
increase). This difference was significant using a z-test of proportions for two independent 
samples with unpooled variance (z = 2.80, p < .05). Likewise, on the DIBELS ISF, the KITS 
group showed a significantly greater percentage increase (8%) in the number of children in the 
low-risk group than did the SAU group (2%; z = 2.00, p < .05), and the same was true on the 
CAPT, with a larger percentage increase for the KITS group (6%) than for the SAU group (0%) 
in children in the upper 7 stanines (z = 3.00, p < .05). For the PLRS, we examined how many 
children met the critical score for their age in each group. In the KITS group, 61% of the children 
met this benchmark whereas in the SAU group only 52% did (not a significant difference). 


Discussion 
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Children with developmental disabilities and delays face a number of academic, social, 
and behavioral challenges during the transition to kindergarten. To the extent that some of these 
can be prevented, the children’s need for further special education services and intervention by 
school psychologists, counselors, and other school personnel may be reduced, leading to 
increased academic and social success for the children and cost savings for schools. Thus, 
increasing the school readiness of children with developmental disabilities and delays could 
potentially have widespread impacts. 

This study examined the efficacy of a short-term intervention at boosting the early 
literacy skills of children who had developmental disabilities and delays and co-occurring 
behavioral difficulties. The KITS Program was designed to fill the summer services gap just 
before kindergarten and provide school readiness groups focused on early literacy, social, and 
self-regulatory skills, as well as parent groups to increase awareness of school readiness and later 
involvement in school. The KITS intervention had a significant positive effect on children’s 
overall early literacy skills (phonemic awareness, letter knowledge, and understanding of 
concepts about print). Further, there was a significant indirect path from the intervention to 
kindergarten teacher ratings through early literacy skills at the end of the summer. In short, the 
intervention improved kindergarten skills by enhancing pre-kindergarten literacy. 

The intervention increased the percentage of children who were at low risk for difficulties 
in reading as measured by standard benchmarks on the DIBELS that appear to be valid 
predictors of later reading acquisition (Burke et al., 2009). Thus, for some of the children, the 
intervention may prevent future difficulties with reading which could mean cost savings for 
schools on remedial services. However, a number of children remained at risk for reading 


difficulties based on kindergarten fall benchmarks. This suggests a need for a greater focus on 
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early literacy skills in ECSE programs. This could be difficult as the services may of necessity 
focus on more immediate behavioral or social needs. Such a focus may at least partially explain 
the finding that although children in the KITS groups received significantly more individual 
service hours prior to their enrollment in the study, intervention effects on literacy skills could 
not be attributed to the greater hours of service. The effects of KITS remain significant even 
when individual services were accounted for in the model. 

As was predicted, the effect sizes for the multivariate models are admittedly modest when 
compared to intervention effects across all of the behavioral sciences (Cohen, 1988). However, 
researchers have recently noted that empirical benchmarks based upon interventions similar to 
those being used in a given study might be more meaningful (Hill, Bloom, Black, & Lipsey, 
2008; Lipsey et al., 2012). When we compare the effect size found in the present study to the 
median effect size of .19 demonstrated in the meta-analysis of summer interventions described 
above (Cooper et al., 2000), they are similar in magnitude. The majority of the studies included 
in the meta-analysis were conducted with typically developing children. 

Very few studies have examined the effects of summer interventions for children with 
developmental delays and disabilities. One recent study of a summer reading intervention in the 
general population compared effect sizes for typically developing children to those of children 
receiving special education services and found that effect sizes were typically lower for children 
in special education (Zvoch & Stevens, 2015). Thus, the difference between the effect sizes 
reported here and the median of .19 in the meta-analysis may be reflective of differences in 
populations studied. 

While the children in the KITS intervention showed relative increases in literacy skills, in 


the services-as-usual group the percentages of children at low risk for reading difficulties 
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remained fairly static or decreased slightly. In the few other studies on this topic, children with 
developmental disabilities show significant loss of skills over the summer break (Allinder & 
Eicher, 1994). Given the dearth of information about how to stem such loss in these very high 
needs students, even modest positive findings from a summer intervention focused on preparing 
these children represent hopeful progress. They also suggest that school psychologists working 
with students with developmental disabilities and behavior problems should consider summer 
programming, particularly programs focused on preparing children with the skills that they will 
need for the transition into kindergarten, to help these children be successful in school. 

A number of limitations of the current study should be acknowledged. First, the children 
represented a heterogeneous group with differing types and degrees of developmental disabilities 
and delays. While this is an accurate reflection of the heterogeneity in the population of children 
receiving ECSE services (Scarborough, Hebbeler, Spiker, & Simeonsson, 2011), analyses of 
skills by disability type are precluded by the sample size. Second, the sample represented 
children who had behavioral difficulties and thus the results may not be generalizable to the 
entire population of children with developmental disabilities and delays. As this subpopulation is 
at the highest risk for difficulties in school, this was an important group on which to focus. Third, 
even though children who received the KITS intervention showed gains in early literacy, it might 
have been possible to show larger effects if the program had been focused exclusively on literacy 
or had employed an even smaller group format. As is noted above, this program had three foci 
because the main interest was in increasing all of the skills that children would need to make the 
transition to kindergarten. Finally, it would have been ideal to have directly measured the 
children’s skills at all three timepoints, rather than relying on teacher reports at the kindergarten 


fall assessment point. However, the study design was based on practical necessities and the 
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desire to minimize the burden to participating families. Teacher reports on the PLRS have been 
shown to be valid measures of concurrent child skills (Cabell et al., 2009). 

Overall, results showed that the KITS Program increased the early literacy skills, and 
thus the school readiness, of children with developmental disabilities and delays who also had 
behavioral difficulties that were likely to interfere with the transition to kindergarten. This 
appeared to influence the children’s reading and writing skills in kindergarten, showing that 
gains over the summer before the transition can translate into better school performance. This is 
not to suggest that this or other programs should be considered to be single solutions for 
preventing negative outcomes for these children. Given the challenges faced by many of the 
children, they are likely to require a number of services across time. Findings from the present 
study suggest that the KITS intervention can be a promising option to be used in concurrence and 
sequence with other programs. 

From a practical standpoint, this study suggests that practitioners look towards providing 
these children with a boost to their school readiness skills prior to the kindergarten transition. 
School psychologists would have to identify incoming kindergarteners in the spring who had 
developmental disabilities and behavioral problems. However, most ECSE programs engage in 
transition planning with children’s elementary schools. A related advantage is that by having the 
children in a summer program, school psychologists and other school personnel would be able to 
collect information about the children prior to the start of the school year when the rush of 
incoming students might make it more difficult. Summer school readiness programming has the 
potential to improve the early academic skills of children with developmental disabilities and 
behavior problems and this may contribute to more positive academic and behavioral trajectories 


across the school years. 
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Table 1. Demographic Characteristics of Children and Families in the KITS and SAU Groups 


KITS Group SAU Group 
Mean child age in years 5.26 (0.29) 5.28 (0.28) 
Child sex - % male a 77 
Child ethnicity (%) 
European American 71 67 
Latino 14 14 
African American 1 2 
Native American 1 2 
Asian American 1 1 
Mixed race 12 14 
Child disability category (%) 
Developmental delay 61 58 
Communication delay 29 32 
Autism 9 10 
Orthopedic impairment 1 0 
Final behavioral risk score 2.34 (1.13) 2.34 (1.14) 
Mean Critical Events subscale score 1.33 (1.83) 1.09 (1.54) 
Mean Aggressive Behaviors subscale score 16.81 (7.21) 16.23 (6.61) 
Mean Maladaptive Behaviors subscale score 20.41 (8.57) 19.79 (8.07) 
Median caregiver education => 1 yr of 4 yr < 1 yr of 4 yr 
college college 


Median annual household income 


$30,000 - $39,999 


$25,000 - $29,999 


Note. Values in the parentheses represent standard deviations. 


4] 


Table 2. Means and Standard Deviations for the Indicator and Control Variables by Group 


KITS Group SAU Group 
Tl T2 T3 Tl T2 T3 
M SD M SD M SD M SD M SD M SD 

DIBELS Letter Naming Fluency 7.50 9.79 9.89 12.33 -- -- 8.34 11.39 8.15 11.59 -- -- 
DIBELS Initial Sound Fluency 4.55 5.38 5.61 6.61 -- -- 4.28 4.37 5.63 5.69 -- -- 
Concepts About Print 4.96 3.32 6.26 3.52 -- -- 5.21 3.63 6.04 3.66 -- -- 
Parent ratings of prereading 

skills -0.02 0.85 0.03 0.76 -- -- 0.02 0.83 -0.04 0.80 - -- 
Teacher rating of emergent 

reading -- -- -- -- 3.10 0.68 -- -- -- -- 2.96 0.71 
Teacher rating of emergent 

writing -- -- -- -- 2.60 0.73 -- -- -- -- 2.51 0.72 
WPPSI-III Block Design 20.19 5.88 21.36 5.85 
Hours of individual ECSE 

services 71.24 93.26 49.64 47.67 


Hours of ECSE 


consultation/training/services 11.15 17.76 13.84 34.86 


Table 3. Correlations Among the Indicator Variables 
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1 2, 3 4 5 6 fl 10 11 12 

1. T1 DIBELS Letter Naming 

Fluency an 
2. T1 DIBELS Initial Sound Fluency 27% -- 
3. T1 Concepts About Print 56% .36* -- 
4. T1 Parent ratings of prereading 

skills 5* .13*  .40* -- 
5. T2 DIBELS Letter Naming 

Fluency .80* i23*° .52* 49% -- 
6. T2 DIBELS Initial Sound Fluency .33* A5* .46* .22* .36* -- 
7. T2 Concepts About Print 32* 39*  .70* 537% 2* 44* -- 
8. T2 Parent ratings of prereading 

skills 33" .16* .40* .77* 3% .24* BI 


Table 3. (continued) 
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1 2 3 4 5 6 a 8 9 10 11 12 

9. T3 Teacher rating of emergent 

reading 49% 07 48* .46* 50* .26* 42* .53* -- 
10. T3 Teacher rating of emergent 

writing 37% .15* 39% i 40* 23* 40* 45* .71* -- 
11. WPPSI-III Block Design 28% 14* 40* 25* .21* .16* 34% .26* .19* 33*  -- 
12. Hours of individual ECSE 

services -.08 -.01 -.05 -.11 -.02 -.01 -.05 -.11 -.11 -.12*  -.09 -- 
13. Hours of ECSE 

consultation/training services 12 -.09 -.03 05 07 -.05 -.08 .O1 .03 .O1 -.05 .13 


*p < OS 


Table 4. Percentages of Children Scoring in Low Risk Range for Early Literacy Variables by 
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Group 
KITS Group SAU Group 
Tl 72 Tl T2 
DIBELS LNF 37% 44% 36% 32% 
DIBELS ISF 21% 29% 21% 23% 
CAPT 71% 13% 13% 13% 


Figure 1. Flow of participants through recruitment and randomization 
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recruitment 


= 392 
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n=18 
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n= 225 
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KITS group SAU group 
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Participated in Participated in 
assessments assessments 
n= 107 n= 102 
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Figure 2. Model of Effects of the Intervention on Early Literacy Skills and Indirect Effects on Kindergarten Literacy Skills 


(Indirect path in bold) 
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